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ABSTRACT 


Access  control  to  sensitive  information  is  a  vital  concern  for 
Department  of  Defense  agencies.  Current  methods  employed  to 
control  access  are  vulnerable  to  unauthorized  users  and  frequently 
inadequate.  The  use  of  biometric  access  control  devices,  such  as 
signature  verification  systems,  may  represent  a  solution  to  the 
access  control  problem.  This  thesis  looked  at  two  dynamic 
signature  verif icatior.  systems  and  compared  their  performance  in 
general  as  well  as  under  the  different  operating  conditions  of 
lined  and  unlined  paper  and  morning  and  afternoon  use.  The  two 
signature  verification  systems  were  the  CIC  system  and  the  Sign/On 
system.  Additionally,  the  thesis  compared  the  CIC  system  under 
both  sets  of  operating  conditions  using  an  inking  stylus  pen  and  a 
non-inking  stylus  pen.  The  experimental  results  indicated  there 
was  no  significant  difference  between  the  CIC  system  using  an 
inking  stylus  pen  and  the  Sign/ On  system  and  that  both  systems  had 
Type  I  error  rates  of  less  than  3%  and  Type  II  error  rates  of  less 
than  1%.  The  results  also  indicated  that  the  operating  conditions 
test  did  not  favor  either  system. 
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I .  INTRODUCTION 


Access  control  to  sensitive  information  is  a  vital  concern 
for  Department  of  Defense  agencies.  Because  the  cost  of 
unauthorized  access  is  often  intolerable,  the  techniques  used 
to  control  both  physical  access  and  computer  access  must  be 
closely  examined.  In  the  interest  of  national  security  as 
well  as  personal  privacy  protection,  adequate  safeguards 
against  unauthorized  access  must  be  guaranteed. 

Access  control  is  the  security  function  whereby  a  user  is 
identified  and  screened  to  discriminate  authorized  from 
unauthorized  use  (MIL-HNBK-1031/1,  1987,  p.  114).  There  are 
two  different  types  of  personal  identification  and  screening 
involved  in  access  control  (FIPS  Pub  83,  1980,  p.  9).  The 
first  of  these  is  the  actual  establishment  of  identity.  In 
this  case  it  is  the  controlling  system  which  is  responsible 
for  the  identification  of  the  person  wishing  access.  The 
other  alternative  is  to  simply  ve..ify  a  claimed  identity.  A 
potential  user  would  enter  a  claimed  identity  and  then  perform 
other  system  related  tasks  to  verify  the  claim.  Access  would 
be  granted  or  denied  based  upon  the  verification  of  identity 
rather  than  on  the  establishment  of  identity.  Typically  it  is 
much  faster  and  easier  to  verify  an  identity  rather  than 
establish  an  identity. 
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There  are  three  basic  methods  used  to  verify  a  person's 
identity  for  access  control  (FIPS  Pub  48,  1977,  p.  7).  These 
methods  are: 

1.  Something  a  person  knows. 

2.  Something  a  person  possesses. 

3.  Something  about  a  person. 

The  first  method  relies  on  such  things  as  passwords,  lock 
combinations,  or  other  pieces  of  personal  information  to 
verify  identity.  The  most  common  is  the  password.  There  are 
problems  with  this  method  in  that  the  authorized  user  may 
forget  the  known  information  and  be  denied  access  or  an 
unauthorized  user  may  obtain  the  known  information  and  be 
granted  access.  In  either  situation,  access  control  was  not 
effective. 

The  second  verification  method  relies  on  such  things  as 
keys,  badges,  or  passes.  The  degree  of  security  provided  by 
this  method  is  limited,  however,  because  possessions  can  be 
lost,  stolen  or  duplicated.  Additionally,  accounting  for  each 
key  or  badge  can  be  an  administrative  burden  in  the  ever 
changing  environment  usually  found  in  the  Department  of 
Defense . 

Because  of  the  inherent  vulnerabilities  in  the  other  two 
methods,  physiological  attributes  are  being  considered  as 
possible  bases  for  personal  identification.  Using 
physiological  attributes  for  personal  identification  is  one 
area  in  the  field  of  study  called  biometrics.  Some  of  the 
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physical  attributes  currently  considered  for  access  control 
are  voice,  retinal  blood  vessel  patterns,  signature  dynamics, 
typing  dynamics,  three  dimensional  hand  geometry,  and 
fingerprints.  The  critical  problem  with  biometric  devices 
used  for  identity  verification  is  the  difficulty  in  performing 
precise,  repeatable  measurements  on  the  human  body  (FIPS  Pub 
48,  1977,  p.  10).  This  is  especially  true  for  the  class  of 
biometric  devices  which  require  a  dynamic  measure  like  speech 
or  handwriting  rather  than  a  relatively  static  measure  like  a 
fingerprint.  Another  concern  with  biometric  devices  is  the 
amount  of  interpersonal  variation  versus  intrapersonal 
variation  (FIPS  Pub  83,  1980,  p.  9).  Interpersonal  variations 
are  those  exhibited  among  different  individuals  whereas 
intrapersonal  variations  are  those  exhibited  from  the  same 
individual  from  one  measurement  to  the  next.  Introducing 
intrapersonal  variations  requires  a  much  more  sophisticated 
biometric  device  in  order  to  verify  identity. 

Signature  verification  machines  are  one  type  of  biometric 
device  being  considered  for  access  control.  Handwriting 
generally  presents  unique  characteristics  from  one  individual 
to  the  next.  The  signature,  however,  is  even  more  unique  and 
personally  stylized  because  of  a  lifetime  of  practice.  Often 
the  signature  is  similar  to  a  conditioned  reflex  and  is 
written  with  little  attention  paid  to  each  individual  letter 
(FIPS  Pub  48,  1977,  p.  11).  The  end  result  of  this 
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combination  of  features  produces  a  highly  individualized 
biometric  measure  which  is  very  difficult  to  duplicate. 

There  are  two  basic  methods  used  in  signature  verification 
systems  (FIPS  Pub  83,  1980,  p.  33).  One  of  these  involves  the 
comparison  of  a  static  signature.  An  example  is  a  scanning 
device  which  measures  only  the  written  form  of  the  signature. 
This  method  is  extremely  vulnerable  to  deceit.  Another  method 
compares  a  dynamic  signature  with  a  reference  signature.  In 
other  words,  the  potential  user  submits  a  "live”  sample 
signature  which  is  compared  to  a  reference  signature.  What 
this  system  measures  has  nothing  to  do  with  the  signature's 
appearance,  but  with  the  forces  generated  by  the  user  with  the 
stylus  or  writing  instrument  (Bakke,  1986,  p.  113)  This 
method  takes  full  advantage  of  how  the  signature  is  created. 
It  is  much  more  difficult  to  duplicate  the  dynamic  motions 
used  to  create  a  signature  than  it  is  to  duplicate  the 
finished  product. 

There  are  many  ways  to  obtain  the  signals  which  represent 
a  dynamic  signature.  The  quantities  measurable  during  the 
writing  process  are  positions,  forces,  and  accelerations. 
Instrumentation  designed  to  measure  these  quantities  may 
involve  the  writing  instrument  (stylus)  or  writing  surface 
(platen)  or  a  combination  of  the  two  (FIPS  Pub  83,  1980,  p. 
33).  The  vector  of  these  "live"  measurements  is  compared  to 
the  reference  signature  vector  to  verify  identity.  In  theory, 
other  written  words  may  be  used  instead  of  the  signature  to 
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verify  identity,  however,  signatures  are  preferable  because  of 
the  level  of  conditioned  response  involved. 

There  are  two  dynamic  signature  verification  systems  in 
the  Man-Machine  Systems  Design  Laboratory  at  the  Naval 
Postgraduate  School  in  Monterey,  California.  One  of  them  is 
produced  by  the  Communication  Intelligence  Corporation  and  is 
called  On-line  Dynamic  Signature  Verification.  This  system 
consists  of  the  stylus,  the  platen,  and  the  software 
algorithms.  Currently  the  system  is  connected  to  a  personal 
computer  for  operation.  The  other  system  is  produced  by 
Capital  Security  Systems  Incorporated  and  is  called  Sign/On. 
This  system  consists  of  the  stylus,  the  platen,  and  a 
controlling  unit  with  the  software  algorithm.  The  Sign/On  is 
connected  to  a  monitor  for  use.  The  purpose  of  this  thesis  is 
to  design  an  experiment  to  evaluate  and  compare  these  two 
systems,  analyze  the  statistical  results,  and  report  the 
findings. 
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II.  THE  EXPERIMENT 


This  experiment  compared  the  effectiveness  of  two 
signature  verification  systems  under  different  operating 
conditions.  One  of  the  systems  was  produced  by  the 
Communication  Intelligence  Corporation  (CIC)  and  was  called 
On-line  Dynamic  Signature  Verification.  The  other  signature 
verification  system  was  produced  by  Capital  Security  Systems 
Incorporated  and  was  called  Sign/On. 

The  different  operating  conditions  being  compared  were 
lined  paper  versus  unlined  paper,  morning  versus  afternoon 
use,  and  an  inking  stylus  versus  a  non-inking  stylus  pen. 
Only  the  CIC  system  was  capable  of  operating  with  a  non-inking 
stylus  and  therefore,  the  inking  stylus  pen  versus  the  non¬ 
inking  stylus  pen  condition  was  compared  for  that  system 
alone.  The  comparison  of  these  two  systems  under  the 
different  operating  conditions  may  suggest  preferred  operating 
conditions  for  better  performance  and  may  indicate  a  more 
effective  system. 

A.  THE  EQUIPMENT 

Both  the  Communication  Intelligence  Corporation  and 
Capital  Security  Systems  Incorporated  were  contacted  to 
request  specific  information  concerning  the  measurements 
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recorded  for  verification.  Neither  company  chose  to  respond 
to  the  requests.  Operating  procedures  for  both  systems  are 
described  below.  However,  generic  information  provided  by  the 
companies  indicated  each  system  works  on  the  speed  of 
acceleration  and  deceleration  of  the  stylus  movements  and  not 
on  the  pressure  applied  to  the  stylus. 

1.  The  CIC  System 

When  first  approaching  the  CIC  system  the  usev  is 
prompted  by  the  monitor  screen  to  select  a  mode  of  operation 
from  the  main  menu.  The  choices  are  either  enroll  or  verify. 
The  user  touches  the  stylus  to  the  enroll  or  verify  box 
(Figure  1)  depending  on  the  user's  wishes.  The  monitor  screen 
then  prompts  the  user  to  enter  his  personal  identification 
number  (PIN)  in  the  scratchpad.  The  user  must  print  each  of 
the  alphanumeric  characters  in  a  separate  box  in  the 
scratchpad.  The  monitor  screen  echoes  the  input  from  the 
scratchpad.  The  system  must  correctly  identify  the 
characters  in  order  to  retrieve  the  correct  file.  If  a 
character  recognized  by  the  system  is  incorrect,  the  user 
simply  writes  the  character  over  in  the  same  box  until  it  is 
recognized  correctly.  Once  the  PIN  is  correctly  recognized, 
the  monitor  screen  prompts  the  user  to  touch  the  continue  box 
on  the  writing  surface  with  the  stylus  pen  for  faster 
response.  The  system  now  retrieves  the  proper  file  and 
signature  verification  can  begin. 
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The  user  is  now  prompted  by  the  monitor  screen  to 
submit  a  signature.  As  before  the  user  is  asked  to  touch  the 
continue  box  for  faster  response.  After  completion  of  the 


Figure  1.  CIC  SYSTEM  WRITING  SURFACE 


CIC  SIGNATURE  VERIFICATION  SYSTEM 


REJECT  I 


signature,  the  system  compares  the  submitted  signature  to  the 
reference  signature  for  verification.  Depending  upon  the 
comparison,  the  system  may  either  use  the  monitor  screen  to 
prompt  the  user  for  up  to  two  more  signatures  or  make  a 
verification  decision.  The  system  will  prompt  the  user  for 
another  signature  if  the  comparison  of  the  signatures  is 
"close”  but  not  exact.  However,  after  a  total  of  three 
submitted  signatures  a  verification  decision  must  be  made.  If 
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the  comparison  is  vastly  different  or  exact,  the  verification 
decision  can  be  made  immediately.  The  results  of  the  decision 
will  be  displayed  on  the  monitor  screen.  The  monitor  screen 
will  show  the  user  that  the  system  accepted  the  claimed 
identity  or  that  the  system  recognized  a  forgery.  At  this 
time  the  user  touches  the  quit  box  with  the  stylus  to  return 
to  the  main  menu. 

2.  The  Sign/On  System 

When  first  approaching  the  Sign/On  system,  the  monitor 
screen  and  an  illuminated  light  above  ENTER  ID  on  the  writing 
surface  prompt  the  user  for  a  PIN  (Figure  2) .  The  user  types 
the  PIN  on  the  monitor's  keyboard.  The  writing  surface  has  a 
series  of  red  lights  to  prompt  the  user  from  this  point  until 
completion  of  the  access  attempt.  After  the  PIN  has  been 
entered,  the  light  above  SIGN  ON  on  the  writing  surface  is 
illuminated.  The  user  then  submits  a  signature  on  the  tablet. 
Based  on  a  single  signature  the  verification  decision  is  made. 
If  the  system  verified  the  claimed  identity,  the  DOOR  OPEN 
light  illuminates.  If  the  system  rejects  the  claimed 
identity,  all  the  lights  above  the  writing  surface  flash. 
Additionally,  the  monitor  screen  duplicates  the  lighted 
verification  decision.  The  system  is  then  ready  to  prompt  the 
next  user  for  a  PIN. 
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Figure  2  .  SIGN/ON  SYSTEM  IfRITING  SURFACE 


B.  PERFORMANCE  MEASURES 

The  purpose  of  the  experiment  was  to  compare  the 
performance  of  these  two  signature  verification  systems  under 
different  operating  conditions.  In  order  to  accurately 
measure  performance,  two  types  of  errors  could  be  identified 
(Holmes,  1990,  p.  2).  They  were: 

1.  Type  I  Errors:  Falsely  rejecting  an  authorized  veer. 

2.  Type  II  Errors:  Falsely  accepting  an  unauthorized 

user. 

The  perfect  biometric  device  can  positively  distinguish 
interpersonal  variances  from  intrapersonal  variances  and  thus 
eliminate  both  Type  I  and  Type  II  errors.  In  other  words  if 


the  biometric  device  can  accurately  identify  intrapersonal 
variances,  the  result  is  a  decline  in  the  number  of  Type  I 
errors.  Similarly,  if  the  device  can  correctly  recognize 
interpersonal  variations.  Type  II  errors  decrease.  However, 
if  the  system  is  not  perfect,  it  must  adequately  deal  with 
both  types  of  errors. 

C.  SUBJECTS 

In  all,  24  subjects  were  used  in  the  e^.periment.  They 
were  all  military  officers  at  the  Naval  Postgraduate  School 
(NPS)  in  Monterey,  California.  All  were  pursuing  graduate 
degrees  at  the  time  of  the  testing.  There  were  5  females  and 
19  males  ranging  in  age  from  27  to  44  participating.  None 
were  familiar  with  signature  verification  equipment  prior  to 
testing  and  no  incentives  were  given  for  participation.  The 
two  systems  were  set  up  in  the  Man-Machine  Systems  Design 
Laboratory  at  NPS  and  were  available  to  the  subjects  from  0800 
to  1700  each  weekday.  Testing  was  unsupervised  to  create  a 
more  realistic  environment  and  data  sheets  recording  the 
success  or  failure  of  each  access  attempt  were  collected  each 
week  of  testing.  The  subject  sample  was  considered 
representative  of  the  population  of  military  members  requiring 
access  to  controlled  areas  and  information. 
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D.  ENROLLMENT 


The  first  part  of  the  experiment  was  the  enrollment 
process.  This  process  was  the  procedure  by  which  the 
signature  verification  system  converts  a  subject's  signature 
into  a  reference  template.  A  reference  template  is  a  sample 
of  the  subject's  signature  stored  in  memory  to  which  an  actual 
signature  is  compared  when  access  is  attempted.  If,  based  on 
this  comparison,  the  system  verifies  the  subject's  identity, 
access  will  be  granted.  Otherwise,  the  subject  will  be  denied 
access . 

Each  subject  was  guided  through  the  enrollment  procedure 
by  the  experimenter.  For  the  CIC  system  each  subject  was 
assigned  a  three  letter  PIN.  The  first  action  required  by 
each  subject  was  to  select  the  enroll  mode.  This  was  done  by 
touching  the  stylus  pen  to  the  enroll  box  on  the  system 
writing  surface  (Figure  1) .  Next  the  subject  was  prompted  to 
enter  his  PIN  in  the  system  scratchpad.  The  subject  was  then 
prompted  to  write  six  signatures.  After  each  signature  the 
subject  was  given  the  choice  to  submit  or  reject  the 
signature.  Six  submitted  signatures  were  required  for 
enrollment.  The  monitor  then  prompts  the  subject  for  one  more 
signature  to  verify  enrollment.  At  this  time  the  signature 
template  was  created  and  stored  in  memory.  To  test  the 
accuracy  of  the  enrollment  procedure  and  to  familiarize  the 
subject  with  the  system,  each  subject  was  required  to  gain 
access  three  consecutive  times.  If  any  of  these  three 
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attempts  were  unsuccessful,  the  subject  repeated  the 
enrollment  procedure  from  start  to  finish  until  three 
successful  attempts  to  gain  access  were  recorded. 

The  enrollment  procedure  for  the  Sign/On  system  was  also 
guided  by  the  experimenter.  Each  subject  must  again  create  a 
reference  template.  To  create  the  template  on  the  Sign/On 
system  each  subject  first  entered  an  assigned  three  digit  PIN 
on  the  monitor  keypad.  The  system  illuminates  the  light  above 
SIGN  ON  on  the  writing  surface  (Figure  2)  to  prompt  the 
subject  to  submit  two  signatures.  Thus  the  template  was 
created  and  stored  in  the  system's  memory.  Again  each  subject 
was  required  to  gain  access  three  consecutive  times  to  become 
familiar  with  the  system  and  to  test  the  accuracy  of  the 
enrollment  procedure. 

Both  systems  used  lined  paper  and  inking  stylus  pens  for 
enrollment.  Half  the  subjects  were  enrolled  in  a  morning 
session  and  the  other  half  in  an  afternoon  session.  An 
enrollment  procedure  similar  to  the  one  described  above  might 
be  appropriate  for  new  users  in  an  operational  environment. 

E.  TESTING  PROCEDURES 

The  experiment  ran  for  a  period  of  10  weeks  following  the 
enrollment  process.  it  was  conducted  in  4  phases.  Phase  1 
consisted  of  access  attempts  on  both  systems  testing  the 
effects  of  lined  and  unlined  paper  and  morning  and  afternoon 
use.  The  CIC  system  allows  the  subject  to  write  his  signature 
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up  to  three  times  before  access  is  granted  or  denied. 
However,  if  the  signature  is  vastly  different  from  the 
template  on  either  of  the  first  two  signatures,  access  is 
immediately  denied.  In  other  words  because  the  first 
signature  is  vastly  different  from  the  template  stored  in  the 
CIC  system's  memory,  three  signatures  are  not  permitted  and 
access  is  immediately  denied.  With  the  Sign/On  system  the  user 
is  allowed  only  one  signature  after  which  a  verification 
decision  is  made.  In  order  to  compare  the  systems  for  this 
experiment,  however,  the  Sign/On  system  was  adjusted  so  that 
the  subject  could  attempt  access  with  up  to  three  signatures. 
A  success  on  either  system  was  defined  to  be  access  granted  on 
any  of  three  signatures. 

The  subjects  were  asked  to  attempt  access  four  times  per 
day  on  each  system  both  in  the  morning  and  in  the  afternoon 
for  three  weeks.  In  total  16  signatures  were  collected  from 
each  subject  each  day.  The  systems  were  alternated  between 
lined  and  unlined  paper  on  a  daily  basis.  The  morning  test 
period  ran  from  0800  to  1200  and  the  afternoon  period  ran  from 
1201  to  1700.  The  systems  were  available  for  four  days  per 
week  to  facilitate  both  paper  conditions  (lined  and  unlined 
paper) .  Subjects  were  asked  to  alternate  their  first  access 
attempt  each  day  between  the  two  systems  to  eliminate  any 
possible  effects  due  to  ordering. 

Phase  2  of  the  experiment  concerned  falsely  accepting 
unauthorized  users,  or  Type  II  errors.  This  is  also  called 
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impostor  testing.  A  list  of  subject  identification  codes  for 
both  systems  as  well  as  a  sample  signature  was  posted.  Each 
subject  was  asked  to  attempt  access  using  every  other 
subject's  signature.  The  impostor,  or  subject  using  another's 
identification  code,  attempted  access  on  lined  paper  in  the 
morning  and  afternoon  and  on  unlined  paper  in  the  morning  and 
afternoon.  Each  impostor  attempted  access  in  every  other 
account  one  time  for  both  time  conditions  and  both  paper 
conditions.  Access  was  granted  or  denied  on  both  systems 
based  on  up  to  three  signatures  per  impostor  depending  on  the 
comparison  between  the  submitted  signature  and  the  reference 
template.  Because  of  the  difficulty  in  imitating  another's 
signature,  it  was  assumed  the  differences  between  the  two 
systems  in  the  number  of  signatures  allowed  before  access  was 
granted  or  denied  would  not  matter.  More  specifically,  the 
impostor  signature  and  the  reference  template  were  assumed  to 
be  so  different  that  the  CIC  system  would  make  a  verification 
decision  on  the  first  attempt.  In  total  24  impostor 
signatures  per  subject  for  each  condition  were  collected. 

Phase  3  was  a  replication  of  Phase  1  but  only  using  the 
CIC  system  with  the  non-inking  stylus  pen.  All  subjects  were 
re-enrolled  in  the  CIC  system  as  before.  Lined  paper  was  used 
for  the  re-enrollment  with  half  the  subjects  re-enrolled  in  a 
morning  session  and  the  other  half  in  an  afternoon  session. 
The  experiment  proceeded  as  described  in  Phase  1  with  the  CIC 
system  and  the  non-inking  stylus. 
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Phase  4  of  the  experiment  was  identical  to  Phase  2  using 
the  non-inking  stylus  pen  on  the  CIC  system  alone. 

The  data  collected  from  all  four  phases  of  the  experiment 
was  analyzed  to  compare  performance  of  the  two  signature 
verification  systems.  The  Sign  Test  was  used  to  compare  the 
systems  and  a  confidence  interval  was  used  to  estimate  the 
proportion  of  successful  access  attempts  for  both  systems. 
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III.  RESULTS  AND  DISCUSSION 


After  analyzing  the  data  collected  from  the  experiment, 
the  Sign  Test  was  determined  appropriate  to  compare  (1)  the 
CIC  and  the  Sign/On  signature  verification  systems,  (2)  the 
CIC  signature  verification  system  with  an  inking  and  a  non¬ 
inking  stylus,  (3)  the  CIC  and  the  Sign/On  systems  for 
impostor  testing,  and  (4)  the  CIC  system  with  an  inking  stylus 
and  a  non-inking  stylus  for  impostor  testing.  Two  separate 
series  of  tests  were  completed  for  each  of  the  four  situations 
above.  The  first  series  consisted  of  4  sets  of  four  tests 
each  and  compared  each  system  under  the  different  operating 
conditions  (lined  and  unlined  paper;  morning  and  afternoon 
use) .  The  second  series  consisted  of  4  tests  that  combined 
all  four  operating  conditions  and  re-compared  the  systems. 
In  all  twenty  different  Sign  Tests  were  completed. 
Additionally,  three  confidence  intervals  were  calculated  to 
estimate  the  proportion  of  successful  access  attempts  for  the 
CIC  system  with  an  inking  stylus,  the  Sign/On  system,  and  the 
CIC  system  with  a  non-inking  stylus.  Three  more  confidence 
intervals  were  calculated  to  estimate  the  proportion  of 
successful  access  attempts  during  impostor  testing  for  the  CIC 
system  with  an  inking  stylus,  the  Sign/On  system,  and  the  CIC 
system  with  a  non-inking  stylus. 
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A.  DATA  COLLECTED 

Due  to  unforeseen  scheduling  conflicts,  seven  subjects 
were  unable  to  attempt  access  the  requested  total  of  24  times 
for  each  operating  condition.  These  seven  subjects  did, 
however,  attempt  access  a  total  of  20  times  for  each  operating 
condition.  Because  the  data  used  to  compute  the  statistics 
were  proportions,  the  difference  due  to  sample  size  between  a 
proportion  of  24  samples  and  one  of  20  samples  was  considered 
negligible.  Additionally,  only  18  subjects  participated  in 
the  phases  of  the  experiment  using  the  CIC  system  with  the 
non-inking  stylus.  The  smaller  sample  size  was  used  in  the 
comparison  of  the  CIC  system  with  an  inking  and  a  non-inking 
stylus . 

B.  SIGN  TEST 

The  Sign  Test  is  used  in  testing  to  determine  if  one 
random  variable  in  a  pair  (X,Y)  tends  to  be  larger  than  the 
other  random  variable  in  the  pair  (Conover,  1980,  p.  122)  . 
For  this  experiment,  the  random  variable  pairs  were 


proportions 

of  successful  access  attempts.  The 

basis 

for 

pairing  the 

observations  was 

the  common  subject 

for 

each 

proportion. 

The  proportions 

within  the  pair 

were 

not 

independent. 

however,  each 

bivariate  random 

pair 

was 

independent 

of  the  others. 

In  addition,  the  pairs 

were 

consistent  for  each  separate  test. 
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1.  The  CIC  System  vs.  The  Sign/On  System 


For  the  four  Sign  Tests  comparing  the  different 
operating  conditions  for  the  CIC  system  and  the  Sign/On 
system,  the  X's  were  defined  to  be  the  proportion  of 
successful  access  attempts  on  the  CIC  system  and  the  Y's  were 
defined  to  be  the  proportion  of  successful  access  attempts  on 
the  Sign/On  system.  The  test  and  the  results  from  the  first 
Sign  Test  comparing  the  two  systems  on  lined  paper  in  the 
morning  are  summarized  in  Table  1.  The  test  clearly 
demonstrated  that  the  combined  operating  conditions  of  lined 
paper  and  morning  use  did  not  significantly  favor  one  system 
over  the  other.  The  significance  level  was  0.01. 

The  remainder  of  the  Sign  Tests  and  their  results  are 
in  Tables  2  -  15  in  the  Appendix.  Table  2  summarized  the  test 
and  the  results  from  the  second  Sign  Test  for  the  comparison 
of  the  CIC  system  and  the  Sign/On  system  under  lined  paper  in 
the  afternoon.  Again  the  combination  of  operating  conditions 
was  consistent  among  the  systems  and  did  not  indicate  any 
difference  between  the  two  systems  with  a  significance  level 
of  0.01. 

The  results  from  third  Sign  Test  are  similarly 
summarized  in  Table  3.  No  significant  differences  between  the 
performances  of  the  CIC  system  and  the  Sign/On  system  were 
indicated  by  combining  the  operating  conditions  of  unlined 
paper  and  morning  use.  The  significance  level  was  0.01. 
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Table  1.  SIGN  TEST  FOR  THE  L^C  SYSTEM  vs.  THE  SIGN/ON 
SYSTEM  ON  LINED  PAPER  IN  THE  MORNING 


SUBJECT* 

CIC 

SIGN/ON 

DIFFEI 

1 

1.000 

0.950 

+ 

2 

1.000 

0.917 

+ 

3 

1.000 

0.875 

+ 

4 

1.000 

0.900 

+ 

5 

0.917 

0.958 

- 

6 

1.000 

0.917 

+ 

7 

1.000 

0.958 

+ 

8 

0.958 

1.000 

- 

9 

0.875 

0.958 

— 

The  hypotheses 

;  are: 

Ho:  The  CIC  system  and  the  Sign/On  system  have  the  same 

proportion  of  successful  access  attempts  on  lined  paper 
in  the  morning 

Ha:  The  CIC  system  and  the  Sign/On  system  do  not  have 

the  same  proportion  of  successful  access  attempts  on 
lined  paper  in  the  morning 

a  =  0.01 

The  observed  number  of  minus  signs  is  3  out  of  a 
possible  9.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.2539. 

Because  the  probability  level  is  greater  than  o,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  BOTH  SYSTEMS  ARE  THE  SAME  ON  LINED  PAPER  IN 
THE  MORNING  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  o  were 
considered 


The  final  Sign  Test  comparing  the  CIC  system  and  the 
Sign/On  system  under  the  different  operating  conditions 
compared  the  systems  under  unlined  paper  with  afternoon  use. 
The  test  and  results  are  summarized  in  Table  4.  There  was  no 
significant  difference  between  the  two  systems  at  a 
significance  level  of  0.01  on  unlined  paper  in  the  afternoon. 
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In  summary,  neither  of  the  operating  conditions  of 
paper  nor  time  significantly  favored  either  the  CIC  system  or 
the  Sign/On  system.  It  should  be  noted,  however,  that  because 
the  same  subjects  were  used  in  all  the  Sign  Tests,  the  tests 
and  their  respective  results  are  not  independent. 

2.  The  CIC  System  with  Inking  Stylus  vs.  The  CIC  System 
with  Non-Inking  Stylus 

A  second  set  of  four  Sign  Tests  was  performed  to 
compare  the  different  operating  conditions  for  the  CIC  system 
with  an  inking  stylus  and  the  CIC  system  with  a  non-inking 
stylus.  In  these  four  test  the  X's  were  defined  to  be  the 
proportion  of  successful  access  attempts  using  an  inking 
stylus  and  the  Y's  were  defined  to  be  the  proportion  of 
successful  access  attempts  using  a  non-inking  stylus.  Table 
5  summarized  the  test  and  th-  results  from  the  comparison  of 
the  inking  stylus  with  the  non-inking  stylus  for  the  operating 
conditions  of  lined  paper  and  mri-ning  use.  Table  5  indicates 
that  there  is  no  significant  difference  between  the  two 
systems  using  lined  paper  in  the  morning.  The  significance 
level  was  0.01. 

The  results  from  the  comparison  of  the  CIC  system 
using  both  an  inking  and  a  non-inking  stylus  on  lined  paper  in 
the  afternoon  as  well  as  the  test  itself  are  summarized  in 
Table  6.  The  results  demonstrated  that  there  was  not  a 
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significant  difference  between  the  two  systems  using  lined 
paper  in  the  afternoon  at  a  significance  level  of  0.01. 

The  next  Sign  Test  performed  compared  the  systems  on 
unlined  paper  in  the  morning.  The  test  and  results  are 
summarized  in  Table  7.  These  results  suggest  that  there  was, 
again,  no  difference  among  the  systems  due  to  operating 
conditions.  The  significance  level  was  0.01. 

The  last  Sign  Test  for  this  set  of  four  compared  the 
CIC  system  with  an  inking  stylus  and  a  non-inking  stylus  using 
unlined  paper  in  the  afternoon.  The  test  and  its  results  are 
again  summarized  in  Table  8.  Analyzing  the  results  show  that 
there  was  no  significant  difference  in  the  systems  because  of 
the  operating  conditions.  In  other  words  using  unlined  paper 
in  the  afternoon  did  not  significantly  alter  the  performance 
of  one  system  over  the  other.  The  significance  level  was 
0.01. 

To  summarize  the  results  from  this  set  of  Sign  Tests, 
there  was  no  significant  difference  in  the  performance  of  the 
CIC  system  due  to  the  use  of  an  inking  or  a  non-inking  stylus 
for  the  type  of  paper  and  the  time  of  use. 

3.  The  CIC  System  vs.  The  Sign/On  System  for  Impostor 

Testing 

The  next  set  of  four  Sign  Tests  was  used  to  compare 
the  CIC  system  and  the  Sign/On  system  under  the  different 
operating  conditions  for  the  impostor  testing  portion  of  the 
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experiment.  The  definition  of  a  successful  access  attempt 
changed  for  the  impostor  testing.  A  successful  access  attempt 
was  one  in  which  the  system  successfully  identified  an 
impostor  and  did  not  grant  access.  For  three  of  the  operating 
conditions  none  of  the  impostors  were  granted  access  and, 
therefore,  the  tests  and  their  corresponding  results  were  not 
presented.  This  was  the  case  for  (1)  lined  paper  and  morning 
use,  (2)  lined  paper  and  afternoon  use,  and  (3)  unlined  paper 
and  afternoon  use.  For  the  conditions  of  unlined  paper  and 
morning  use,  two  impostors  were  granted  access  on  the  Sign/On 
system  and  one  impostor  was  granted  access  on  the  CIC  system. 
Table  9  summarized  the  results  and  the  test  for  the  conditions 
of  unlined  paper  and  morning  use.  There  was  not  a  significant 
difference  between  the  two  systems  under  these  conditions. 
The  significance  level  was  0.01. 

As  before,  this  set  of  Sign  Tests  did  not  indicate  a 
significant  difference  in  system  performance  between  the  CIC 
and  the  Sign/On  systems  for  impostor  testing  due  to  lined  or 
unlined  paper  or  due  to  morning  or  afternoon  use. 

4.  The  CIC  System  with  Inking  Stylus  vs.  The  Sign/On 
System  with  Non-Inking  Stylus  For  Impostor  Testing 
The  final  set  of  four  Sign  Tests  comparing  the  CIC 
system  with  an  inking  and  a  non-inking  stylus  for  impostors 
under  the  different  operating  conditions  was  then  performed. 
A  total  of  three  impostors  were  granted  access  on  the  CIC 
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system  with  a  non-inking  stylus.  Two  of  these  impostors  were 
granted  access  in  the  afternoon  on  lined  paper.  The  third 
impostor  was  granted  access  in  the  afternoon  on  unlined  paper. 
The  impostor  who  was  granted  access  on  the  CIC  system  with  an 
inking  stylus  in  the  previous  phase  did  not  participate  in 
this  phase  of  the  experiment  and,  therefore,  for  this  series 
of  Sign  Tests  there  were  no  impostors  granted  access  on  the 
CIC  system  with  an  inking  stylus.  Table  10  summarized  the 
test  and  the  results  for  the  lined  paper  and  afternoon  use  of 
the  impostor  testing.  The  results  indicated  that  there  was  no 
significant  difference  between  the  system  with  an  inking 
stylus  and  a  non-inking  stylus.  The  significance  level  was 
0.01. 

The  Sign  Test  and  its  results  for  the  impostor  testing 
during  the  afternoon  using  unlined  paper  were  summarized  in 
Table  11.  The  results  indicated  that  the  CIC  system  using  an 
inking  stylus  did  not  perform  any  better  than  the  same  system 
with  a  non-inking  stylus.  Again,  the  significance  level  of 
0.01  was  used. 

In  summary,  this  set  of  four  Sign  Tests  for  impostor 
testing  did  not  demonstrate  a  significant  difference  in 
performance  between  the  CIC  system  using  an  inking  or  a  non¬ 
inking  stylus  for  the  paper  conditions  and  the  time 
conditions . 
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5.  Sign  Tests  for  Combined  Operating  Conditions 


By  combining  all  the  operating  conditions  into  a 
single  measure  for  each  system,  an  overall  measure  for  system 
performance  will  be  obtained.  In  total  four  Sign  Tests  were 
completed  in  this  series.  The  first  test  compared  the  CIC 
system  with  an  inking  stylus  and  the  Sign/On  system.  Table  12 
presented  the  test  and  the  results.  Once  again  the  results 
indicated  that  there  was  no  difference  in  the  performance  of 
either  system.  The  significance  level  was  0.01. 

The  next  Sign  Test  for  the  combined  operating 
conditions  compared  the  CIC  system  using  an  inking  stylus  and 
a  non-inking  stylus.  The  test  and  results  are  summarized  in 
Table  13.  At  a  significance  level  of  0.01,  there  was  no 
difference  in  the  performance  of  the  CIC  system  due  to  the 
condition  of  the  stylus. 

Combining  the  operating  conditions  for  the  impostor 
testing  was  next.  Comparing  the  CIC  system  and  the  Sign/On 
system  for  this  Sign  Test  produced  the  results  in  Table  14. 
These  results  indicated  that  there  was  no  significant 
difference  between  the  two  systems  for  impostor  testing.  The 
significance  level  was  0.01. 

The  last  Sign  Test  for  the  combined  operating 
conditions  compared  the  CIC  system  using  an  inking  and  a  non¬ 
inking  stylus  for  the  impostor  testing.  Table  15  summarized 
the  results  from  that  test  as  well  as  the  test  itself.  The 
total  number  of  impostors  granted  access  was  two.  The  results 
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indicated  that  the  CIC  system  with  an  inking  stylus  did  not 
perform  significantly  better  than  the  same  system  with  a  non¬ 
inking  stylus.  The  significance  level  was  0.01. 

C.  CONFIDENCE  INTERVALS 

In  addition  to  the  hypothesis  testing  done  with  the  Sign 
Tests,  confidence  intervals  were  computed  to  estimate  the 
proportion  of  successful  access  attempts  for  the  CIC  system 
with  an  inking  stylus,  the  Sign/On  system,  and  the  CIC  system 
with  a  non-inking  stylus.  The  confidence  intervals  were 
computed  for  both  the  subjects'  access  attempts  and  the 
impostors'  access  attempts.  Because  of  the  large  values  for 
the  proportion  of  successful  access  attempts,  the  Poisson 
approximation  to  the  Binomial  was  used  to  compute  the  three 
confidence  intervals  for  impostor  testing  (Duncan,  1986,  pp. 
100-103) .  Specifically,  the  Cumulative  Probability  Curves  for 
the  Poisson  Distribution  were  used  (Duncan,  1986,  pp.  94,  573- 
574).  The  confidence  intervals  for  the  proportion  of 
successful  access  attempts  during  non-impostor  testing, 
however,  were  computed  using  the  Normal  approximation  to  the 
Binomial  because  the  actual  number  of  authorized  subjects 
denied  access  was  greater  than  30  (Duncan,  1986,  p.  574) .  The 
six  confidence  intervals  were  as  follows: 

1.  CIC  system  with  inking  stylus 

p  =  proportion  of  successful  access  attempts  =  0.9862 
CONFIDENCE  INTERVAL:  Pr(0.981  <  p  <  0.991)  =  0.95% 
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2.  Sign/On  system 
p  =  0.9764 

CONFIDENCE  INTERVAL:  Pr (0.970  <  p  <  0.983)  =  0.95% 

3 .  CIC  system  with  non-inking  stylus 
p  =  0.9824 

CONFIDENCE  INTERVAL:  Pr(0.976  <  p  <  0.989)  =  0.95% 

4 .  CIC  system  with  inking  stylus  for  impostor  testing 
p  =  0.9995 

CONFIDENCE  INTERVAL:  Pr (0.998  <  p  <  1.000)  =  0.95% 

5.  Sign/On  system  for  impostor  testing 
p  =  0.9991 

CONFIDENCE  INTERVAL:  Pr( 0.997  <  p  <  1.000)  =  0.95% 

6.  CIC  system  with  non-inking  stylus  for  impostor  testing 
p  =  0.9974 

CONFIDENCE  INTERVAL:  Pr(0.992  <  p  <  0.999)  =  0.95% 


D.  DEPARTMENT  OF  DEFENSE  GUIDELINES 

The  DOD  has  directed  the  United  States  Air  Force  as  the 
lead  agency  to  study  equipment  used  to  automate  physical 
security  access  control  (Bright,  1987,  p.  33).  Under  that 
tasking,  the  Air  Force  has  provided  guidelines  for  the  use  of 
automated  access  control  devices.  These  guidelines  define  a 
goal  of  0.0%  for  both  Type  I  and  Type  II  errors  for  an  access 
control  system.  However,  the  Air  Force  set  the  minimum 
requirement  for  access  control  systems  at  3.0%  Type  I  errors 
and  1.0%  Type  II  errors  (Bright,  1987,  p.  33).  Both  the  CIC 
system  with  an  inking  stylus  and  a  non-inking  stylus  and  the 
Sign/On  system  exceed  those  minimums  and  provide  effective 
access  control. 
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E.  OTHER  CONSIDERATIONS 


There  are  several  other  factors  besides  performance  which 
impact  an  access  control  device.  Some  of  these  are  the  level 
of  security  required,  the  cost  to  operate  the  system,  the 
administrative  burden,  and  the  convenience  to  the  user. 

The  level  of  security  required  is  probably  the  most 
important  factor  to  consider  after  performance.  Highly 
classified  material  will  obviously  demand  a  more  stringent 
access  control  system  than  personal  privacy  information.  The 
ultimate  responsibility  for  determining  the  level  of  access 
control  required  is  set  by  the  individual  commanding  officer 
(OPNAVINST  5510. IH,  1988,  p.2-5).  He  must  be  assured  by 
performance  measures  that  the  access  control  system  is 
adequate  for  the  level  of  information  sensitivity. 

The  cost  to  operate  a  signature  verification  system  is 
very  low.  After  an  initial  purchase  price,  the  system's 
operating  costs  are  essentially  covered  by  paper  and,  in  the 
case  of  the  CIC  system,  ink.  The  approximate  purchase  price 
of  the  CIC  system  is  $1500  while  for  the  Sign/On  system  the 
purchase  price  is  $1900.  These  prices,  however,  are 
approximate  due  to  the  fact  that  each  individual  system  would 
be  tailored  to  fit  the  security  needs  of  the  user. 

The  administrative  burden  associated  with  access  control 
can  be  considerable.  With  an  access  control  system  relying  on 
devices  such  as  passwords  or  keys  the  record  keeping  can  be 
burdensome,  especially  at  very  large  commands.  Signature 
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verification  systems  on  the  other  hand  are  not  an 
administrative  burden.  Both  the  CIC  system  and  the  Sign/On 
system  provide  quick  and  easy  administrative  maintenance.  The 
bulk  of  administrative  tasks  would  entail  enrolling  new  users 
and  deleting  users  who  no  longer  need  access.  Each  of  these 
tasks  can  be  accomplished  in  a  matter  of  seconds  depending  on 
the  operating  capacity  of  the  computer  the  signature 
verification  system  is  using.  The  only  other  administrative 
concern  is  the  amount  of  storage  needed  for  authorized  users. 
This  concern  is  again  a  function  of  the  operating  capacity  of 
the  computer  the  system  is  using. 

Finally,  user  convenience  is  an  important  factor  for 
access  control.  Signature  verification  systems  are  very  easy 
to  learn  and  use  especially  with  and  effective  enrollment 
procedure. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  results  from  this  experiment  indicated  that  both  the 
CIC  On-line  Dynamic  Signature  Verification  system  using  an 
inking  stylus  and  the  Sign/ On  system  produced  by  Capital 
Security  Systems  Incorporated  were  effective  biometric  access 
control  devices.  The  CIC  system  with  an  inking  stylus  had  a 
Type  I  error  rate  of  1.38%  and  a  Type  II  error  rate  of  0.05%. 
The  Sign/On  system  had  a  Type  I  error  rate  of  2.36%  and  a  Type 
II  error  rate  of  0.09%.  In  other  words  authorized  users  were 
denied  access  1.38%  of  the  time  on  the  CIC  system  and  2.36%  of 
the  time  on  the  Sign/On  system  and  impostors  were  granted 
access  0.05%  of  the  time  for  the  CIC  system  and  0.09%  of  the 
time  for  the  Sign/On  system.  The  small  difference  in  the 
percentages  was  not  statistically  significant.  Both  systems 
exceed  the  minimum  requirements  for  access  control  devices  set 
by  the  United  States  Air  Force  and  as  such  provide  adeguate 
security  measures  for  sensitive  information.  Additionally, 
the  experimental  results  also  indicated  that  the  operating 
conditions  of  lined  or  unlined  paper  and  morning  or  afternoon 
use  did  not  significantly  affect  system  performance.  These 
results  seem  to  support  the  claim  that  the  signature  is 
similar  to  a  conditioned  reflex. 

Department  of  Defense  security  requirements  have 
necessitated  the  need  for  stringent  access  control.  Because 
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of  the  vulnerabilities  of  passwords,  badges,  and  keys, 
signature  verifications  systems  like  the  CIC  system  or  the 
Sign/On  system  can  provide  more  reliable  and  accurate  access 
control  systems  and,  thereby,  reduce  the  current  threat  of 
unauthorized  access.  Although  neither  system  is  perfect,  they 
did  exceed  the  published  minimum  requirements  for  access 
control  and  either  system  is  a  feasible  solution  to  an 
individual  facility's  access  control  problem.  The  choice 
remains  with  the  facility's  commander. 

There  are  some  other  areas  of  concern  to  be  investigated 
prior  to  widespread  implementation  of  signature  verification 
systems  used  for  access  control.  The  first  of  these  is  the 
environment  in  which  the  system  operates.  Signature 
verification  T; stems  may  not  be  appropriate  for  deployable 
military  ’inits.  A  second  concern  is  the  traffic  flow  to  the 
controlled  area  of  information.  The  size  of  the  facility  may 
have  a  decided  effect  on  an  access  control  system.  A  third 
concern  is  a  cost  benefit  analysis.  Finally,  verification  of 
these  experimental  results  in  an  operational  environment  would 
demonstrate  the  effectiveness  of  signature  verification 
systems  for  access  control. 
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APPENDIX 


TeOsle  2.  SIGN  TEST  FOR  THE  CIC  SYSTEM  vs.  THE  SIGN/ON 
SYSTEM  ON  LINED  PAPER  IN  THE  AFTERNOON 


SUBJECT*  CIC 

SIGN/ON 

DIFFERENCE 

1  1.000 

0.950 

+ 

2  0.875 

0.958 

- 

3  0.900 

0.750 

+ 

4  0.950 

0.800 

+ 

5  0.958 

1.000 

- 

6  0.917 

0.958 

- 

The  hypotheses  are: 

Ho:  The  CIC  system  and  the  Sign/On  system  have  the  same 

proportion  of  successful 

access  attempts  on 

lined  paper 

in  the  afternoon 

Ha:  The  CIC  system  and 

the  Sign/On  system 

do  not  have 

the  same  proportion  of  successful  access 
lined  paper  in  the  afternoon 

attempts  on 

o  =  0.01 


The  observed  number  of  plus  signs  is  3  out  of  a  possible 
6.  From  any  Binomial  Distribution  Table,  the 

corresponding  probability  level  is  0.6562. 

Because  the  probability  level  is  greater  than  o,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  BOTH  SYSTEMS  ARE  THE  SAME  ON  LINED  PAPER  IN 
THE  AFTERNOON  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 
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Table  3.  SIGN  TEST  FOR  THE  CIC  SYSTEM  vs.  THE  SIGN/ON 
SYSTEM  ON  UNLINED  PAPER  IN  THE  MORNING 


SUBJECT* 

CIC 

SIGN/ON 

DIFFERENCE 

1 

1.000 

0.958 

+ 

2 

1.000 

0.833 

+ 

3 

1.000 

0.917 

+ 

4 

1.000 

0.958 

+ 

5 

0.958 

0.917 

+ 

6 

0.917 

1.000 

- 

7 

0.917 

0.875 

+ 

8 

1.000 

0.917 

+ 

9 

0.958 

1.000 

- 

10 

1.000 

0.958 

+ 

The  hypotheses  are: 

Ho:  The 

CIC  system  and  the 

Sign/On  system  have 

the  same 

proportion  of  successful 

access  attempts 

on 

unlined 

paper  in 

the  morning 

Ha :  The 

CIC  system  and  the  Sign/On  system 

do 

not  have 

the  same  proportion  of  successful  access  attempts  on 
unlined  paper  in  th^  morning 

a  =  0.01 

The  observed  number  of  minus  signs  is  2  out  of  a 
possible  10.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.0547. 

Because  the  probability  level  is  greater  than  o,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  BOTH  SYSTEMS  ARE  NOT  THE  SAME  ON  UNLINED 
PAPER  IN  THE  MORNING  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 


*  only  subjects  with  differences  not  equal  to  0  were 
considered 


Table  4.  SIGN  TEST  FOR  THE  CIC  SYSTEM  VS.  THE  SIGN/ON 
SYSTEM  ON  UNLINED  PAPER  IN  THE  AFTERNOON 


SUBJECT* 

CIC 

SIGN/ON 

DIFFEF 

1 

1.000 

0.958 

•i- 

2 

0.958 

1.000 

- 

3 

1.000 

0.875 

+ 

4 

0.950 

1.000 

- 

5 

0.958 

1.000 

- 

6 

1.000 

0.958 

+ 

7 

0.958 

0.875 

+ 

8 

1.000 

0.958 

+ 

9 

0.875 

1.000 

- 

10 

0.958 

1.000 

- 

11 

0.958 

1.000 

— 

The  hypotheses  are: 

Ho:  The  CIC  system  and  the  Sign/On  system  have  the  same 
proportion  of  successful  access  attempts  on  unlined 
paper  in  the  afternoon 

Ha:  The  CIC  system  and  the  Sign/On  system  do  not  have 

the  same  proportion  of  successful  access  attempts  on 
unlined  paper  in  the  afternoon 

a  =  0.01 

The  observed  number  of  plus  signs  is  5  out  of  a  possible 
11.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.5000. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  BOTH  SYSTEMS  ARE  THE  SAME  ON  UNLINED  PAPER 
IN  THE  AFTERNOON  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 


*  only  subjects  with  differences  not  equal  to  0  were 
considered 


Table  5.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  INKING  STYLUS 
vs.  NON-INKING  STYLUS  ON  LINED  PAPER  IN  THE  MORNING 

SUBJECT*  INKING  NON-INKING  DIFFERENCE 

1  1.000  0.958  + 

2  0.917  0.830  + 

3  0.958  1.000 

4  0.875  1.000 

The  hypotheses  are: 

Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  have  the  same  proportion  of  successful 
access  attempts  on  lined  paper  in  the  morning 

Ha:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  do  not  have  the  same  proportion  of 
successful  access  attempts  on  lined  paper  in  the  morning 

«  =  0.01 

The  observed  number  of  plus  signs  is  2  out  of  a  possible 
4.  From  any  Binomial  Distribution  Table,  the 

corresponding  probability  level  is  0.6875. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  ON  LINED  PAPER  IN  THE  MORNING  AT 
A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 


Tadale  6.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  INKING  STYUJS 
VS.  NON-INKING  STYLUS  ON  LINED  PAPER  IN  THE  AFTERNOON 


SUBJECT* 

1 

2 

3 


INKING 

0.950 

0.958 

0.917 


NON-INKING 

1.000 

0.833 

1.000 


DIFFERENCE 


The  hypotheses  are; 


Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 
inking  stylus  have  the  same  proportion  of  successful 
access  attempts  on  lined  paper  in  the  afternoon 


Ha:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  do  not  have  the  same  proportion  of 
successful  access  attempts  on  lined  paper  in  the 
afternoon 


a  =  0.01 

The  observed  number  of  plus  signs  is  1  out  of  a  possible 
3.  From  any  Binomial  Distribution  Table,  the 

corresponding  probability  level  is  0.5000. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  ON  LINED  PAPER  IN  THE  AFTERNOON  AT 
A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 


36 


Table  7.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  INKING  STYLUS 
vs.  NON-INKING  STYLUS  ON  UNLINED  PAPER  IN  THE  MORNING 


SUBJECT* 

INKING 

NON-INKING 

DIFFERENCE 

1 

1.000 

0.850 

+ 

2 

1.000 

0.950 

+ 

3 

0.900 

0.792 

+ 

4 

0.958 

1.000 

- 

5 

0.958 

1.000 

- 

The  hypotheses  are: 

Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  have  the  same  proportion  of  successful 
access  attempts  on  unlined  paper  in  the  morning 

Ha:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  do  not  have  the  same  proportion  of 
successful  access  attempts  on  unlined  paper  in  the 
morning 

a  =  0.01 

The  observed  number  of  minus  signs  is  2  out  of  a 
possible  5.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.5000. 

Because  the  probability  level  is  greater  than  o,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  ON  UNLINED  PAPER  IN  THE  MORNING  AT 
A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 


Table  8.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  INKING  STYLOS 
vs.  NON-INKING  STYLUS  ON  UNLINED  PAPER  IN  THE  AFTERNOON 


SUBJECT*  INKING  NON-INKING  DIFFERENCE 

1  0.958  1.000 

2  0.958  0.625  + 

3  0.958  1.000 

4  0.958  1.000 

The  hypotheses  are; 

Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  have  the  same  proportion  of  successful 
access  attempts  on  unlined  paper  in  the  afternoon 

Ha;  The  CIC  system  using  an  inking  stylus  and  a  non¬ 
inking  stylus  do  not  have  the  same  proportion  of 
successful  access  attempts  on  unlined  paper  in  the 
afternoon 

a  =  0.01 

The  observed  number  of  plus  signs  is  1  out  of  a  possible 
4.  From  any  Binomial  Distribution  Table,  the 

corresponding  probability  level  is  0.3125. 

Because  the  probability  level  is  greater  than  o,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  ON  UNLINED  PAPER  IN  THE  AFTERNOON 
AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 
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Table  9.  SIGN  TEST  FOR  IMPOSTOR  TESTING  FOR  THE  CIC  SYSTEM 
vs.  THE  SIGN/ON  SYSTEM  ON  UNLINED  PAPER  IN  THE  MORNING 


SUBJECT 

1 

2 

3 


CIC 

1.000 

1.000 

0.958 


SIGN/ON 

0.958 

0.958 

1.000 


DIFFERENCE 

+ 

+ 


The  hypotheses  are; 


Ho:  The  CIC  system  and  the  Sign/On  system  have  the  same 
proportion  of  successful  access  attempts  for  impostor 
testing  on  unlined  paper  in  the  morning 

Ha:  The  CIC  system  and  the  Sign/On  system  do  not  have 

the  same  proportion  of  successful  access  attempts  for 
impostor  testing  on  unlined  paper  in  the  morning 

a  =  0.01 

The  observed  number  of  minus  signs  is  1  out  of  a 
possible  3.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.5000. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  BOTH  SYSTEMS  FOR  IMPOSTOR  TESTING  ARE  THE 
SAME  ON  UNLINED  PAPER  IN  THE  MORNING  AT  A  SIGNIFICANCE 
LEVEL  OF  0.01. 

*  only  subjects  will  differences  not  equal  to  0  were 
considered 
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Table  10.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  AN  INKING 
STYLUS  vs.  NON-INKING  STYLUS  FOR  IMPOSTOR  TESTING  ON  LINED 
PAPER  IN  THE  AFTERNOON 


SUBJECT*  INKING  NON-INKING  DIFFERENCE 

1  1.000  0.944  + 

2  1.000  0.944  + 

The  hypotheses  are: 

Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  for  impostor  testing  have  the  same 
proportion  of  successful  access  attempts  on  lined  paper 
in  the  afternoon 

Ha:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  for  impostor  testing  do  not  have  the  same 
proportion  of  successful  access  attempts  on  lined  paper 
in  the  afternoon 

o  =  0.01 

The  observed  number  of  minus  signs  is  0  out  of  a 
possible  2.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.2500. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  FOR  IMPOSTOR  TESTING  ON  LINED 
PAPER  IN  THE  AFTERNOON  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 
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Table  11.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  INKING  STYLUS 
vs.  NON-INKING  STYLUS  FOR  IMPOSTOR  TESTING  ON  UNLINED  PAPER 
IN  THE  AFTERNOON 

SUBJECT*  INKING  NON-INKING  DIFFERENCE 

1  1.000  0.944  + 

The  hypotheses  are: 

Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  have  the  same  proportion  of  successful 
access  attempts  for  impostor  testing  on  unlined  paper  in 
the  afternoon 

Ha:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  have  the  same  proportion  of  successful 
access  attempts  for  impostor  testing  on  unlined  paper  in 
the  afternoon 

a  =  0.01 

The  observed  number  of  minus  signs  is  0  out  of  a 
possible  1.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.5000. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  FOR  IMPOSTOR  TESTING  ON  UNLINED 
PAPER  IN  THE  AFTERNOON  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 


Table  12.  SIGN  TEST  FOR  THE  CIC  SYSTEM  vs.  THE  SIGN/ON 
SYSTEM 


SUBJECT*  CIC 

SIGN/ON  DIFFERENCE 

1  1.000 

0.966  + 

2  1.000 

0.990  + 

3  0.934 

0.948 

4  1.000 

0.969  + 

5  0.955 

0.920  + 

6  1.000 

0.960  + 

7  0.989 

0.920  + 

8  0.969 

1.000 

9  1.000 

0.990  + 

10  0.938 

0.927  + 

11  0.979 

0.938  + 

12  0.969 

0.990 

13  0.969 

1.000 

14  0.948 

0.969 

15  1.000 

0.938  + 

The  hypotheses  are: 

Ho:  The  CIC  system  and 

the  Sign/On  system  have  the  same 

proportion  of  successful  access  attempts 

Ha:  The  CIC  system  and  the  Sign/On  system  do  not  have 

the  same  proportion  of 

successful  access  attempts 

a  =  0.01 

The  observed  number  of  minus  signs  is  5  out  of  a 

possible  15.  From  any 

Binomial  Distribution  Table,  the 

corresponding  probability  level  is  0.1509. 

Because  the  probability  level  is  greater  than  o,  one 

CANNOT  REJECT  THAT  THE 

PROPORTION  OF  SUCCESSFUL  ACCESS 

ATTEMPTS  FOR  BOTH  SYSTEMS  ARE  THE  SAME  AT  A  SIGNIFICANCE  i 

LEVEL  OF  0.01. 

*  only  subjects  with 

differences  not  equal  to  0  were 

considered 
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Table  13.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  INKING  STYLUS 
VS.  NON-INKING  STYLUS 


SUBJECT 

1 

2 

3 

4 

5 

6 
7 


INKING 

0.958 

0.989 

1.000 

0.938 

0.979 

0.969 

0.948 


NON-INKING 

0.927 

1.000 

0.989 

0.771 

1.000 

1.000 

1.000 


DIFFERENCE 

+ 

+ 

+ 


The  hypotheses  are: 


Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  have  the  same  proportion  of  successful 
access  attempts 

Ha:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  do  not  have  the  same  proportion  of 
successful  access  attempts 

a  =  0.01 

The  observed  number  of  plus  signs  is  3  out  of  a  possible 
7.  From  any  Binomial  Distribution  Table,  the 

corresponding  probability  level  is  0.5000. 

Because  the  probability  level  is  greater  than  o,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 
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Table  14.  SIGN  TEST  FOR  THE  CIC  SYSTEM  vs.  THE  SIGN/ON 
SYSTEM  FOR  IMPOSTOR  TESTING 

SUBJECT*  CIC  SIGN/ON 

1  1.000  0.989 

2  1.000  0.989 

3  0.989  1.000 

The  hypotheses  are; 

Ho:  The  CIC  system  and  the  Sign/On  system  have  the  same 
proportion  of  successful  access  attempts  for  impostor 
testing 

Ha;  The  CIC  system  and  the  Sign/On  system  do  not  have 
the  same  proportion  of  successful  access  attempts  for 
impostor  testing 

o  =  0.01 

The  observed  number  of  minus  signs  is  1  out  of  a 
possible  3.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.5000. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  BOTH  SYSTEMS  ARE  THE  SAME  FOR  IMPOSTOR 
TESTING  AT  A  SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  differences  not  equal  to  0  were 
considered 


DIFFERENCE 

+ 

+ 


44 


Table  15.  SIGN  TEST  FOR  THE  CIC  SYSTEM  WITH  INKING  STYLUS 
vs.  NON-INKING  STYLUS  FOR  IMPOSTOR  TESTING 

SUBJECT*  INKING  NON-INKING  DIFFERENCE 

1  1.000  0.985  + 

2  1.000  0.971  + 

The  hypotheses  are: 

Ho:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  have  the  same  proportion  of  successful 
access  attempts  for  impostor  testing 

Ha:  The  CIC  system  using  an  inking  stylus  and  a  non¬ 

inking  stylus  do  not  have  the  same  proportion  of 
successful  access  attempts  for  impostor  testing 

a  =  0.01 

The  observed  number  of  minus  signs  is  0  out  of  a 
possible  2.  From  any  Binomial  Distribution  Table,  the 
corresponding  probability  level  is  0.2500. 

Because  the  probability  level  is  greater  than  a,  one 
CANNOT  REJECT  THAT  THE  PROPORTION  OF  SUCCESSFUL  ACCESS 
ATTEMPTS  FOR  THE  SYSTEM  IS  THE  SAME  WHETHER  OR  NOT  AN 
INKING  STYLUS  IS  USED  FOR  IMPOSTOR  TESTING  AT  A 
SIGNIFICANCE  LEVEL  OF  0.01. 

*  only  subjects  with  difference  not  equal  to  0  were 
considered 
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